Papers with average F1
Text or Image? What is More Important in Cross-Domain Generalization Capabilities of Hate Meme Detection Models? (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies show that only the textual component of hateful memes enables the multimodal classifier to generalize across domains while the image component proves highly sensitive to a specific training dataset. |
| Approach: | They propose to use only the textual component of hateful memes to generalize across different domains while the image component is highly sensitive to a specific training dataset. |
| Outcome: | The proposed model performs similarly to hate-meme classifiers in a zero-shot setting, while the introduction of meme’s image captions worsens performance by an average F1 of 0.02. |
Improving Knowledge Base Construction from Robust Infobox Extraction (N19-2)
Copied to clipboard
| Challenge: | Existing knowledge bases are incomplete, resulting in poor answers and incompleteness. |
| Approach: | They propose a method to extract Wikipedia infobox tables to populate an existing KB. |
| Outcome: | The proposed method improves accuracy and completeness of the final KB significantly compared to DBpedia's baseline method. |
Coreference Resolution with Entity Equalization (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to coreference resolution capture the properties of entity clusters and use them in the resolution process. |
| Approach: | They propose an approach that captures entities and uses them in coreference resolution . they propose an "Entity Equalization" mechanism that represents each mention in a cluster . |
| Outcome: | The proposed approach improves the CoNLL-2012 coreference resolution task by 3.6%. |
DIA-HARM: Dialectal Disparities in Harmful Content Detection Across 50 English Dialects (2026.acl-long)
Copied to clipboard
| Challenge: | Current disinformation detection systems are predominantly developed and evaluated on Standard American English (SAE) . however, their robustness to dialectal variation is unexplored. |
| Approach: | They propose a benchmark for evaluating disinformation detection robustness across 50 English dialects . they use multi-value's linguistically-grounded transformations to introduce D-CUBE (Dialectal Disinformation Detection Corpus) |
| Outcome: | The proposed model outperforms zero-shot LLMs in human-written dialects while AI-generated content remains stable. |
SrDetection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models (2026.findings-acl)
Copied to clipboard
Shuaimin Li, Liyang Fan, Zeyang li, Zhuoyue Wan, Yufang Lin, Shiwen Ni, Feiteng Fang, Hamid Alinejad-Rokny, Yuanfeng Song, Kun Jing, Chen Jason Zhang, Min Yang
| Challenge: | Existing methods for evaluating code large language models assume access to proprietary training corpora or use external reference sets with manually tuned, non-generalizable thresholds. |
| Approach: | They propose a framework for self-referential leakage detection for gray-box and black-box settings. |
| Outcome: | The proposed framework improves average F1 by 21.52 points in the gray-box setting and 14.46 points in black-box settings over strong baselines. |
Psycholinguistic Tripartite Graph Network for Personality Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on personality detection from online posts adopts multifarious deep neural networks to represent the posts and builds predictive models in a data-driven manner without the exploitation of psycholinguistic knowledge. |
| Approach: | They propose a psycholinguistic knowledge-based tripartite graph network, TrigNet, which consists of a tripartitic graph network and a BERT-based graph initializer. |
| Outcome: | The proposed graph network outperforms the existing state-of-the-art model by 3.47 and 2.10 points in average F1 on two datasets. |
Zero-shot Script Parsing (2022.coling-1)
Copied to clipboard
| Challenge: | Existing resources cover only a small number of tasks, limiting its practical usefulness. |
| Approach: | They propose a zero-shot learning approach to script parsing which enables us to acquire script knowledge without domain-specific annotations. |
| Outcome: | The proposed model outperforms a previous model with scenario-specific supervision and achieves 68.1/74.4 average F1 for event / participant parsing. |
EERPD: Leveraging Emotion and Emotion Regulation for Improving Personality Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for personality detection ignore the connection between psychological knowledge “emotion regulation” and personality traits. |
| Approach: | They propose to use emotion regulation and emotion features to retrieve few-shot samples and provide process CoTs for inferring labels from text. |
| Outcome: | The proposed method outperforms SOTA by 15.05/4.29 on the two benchmark datasets. |
Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus (2020.coling-main)
Copied to clipboard
| Challenge: | Large text corpora are increasingly important for a wide variety of NLP tasks. |
| Approach: | They propose to train automatic language identification models on up to 1,629 languages . they find that human-judged accuracy for web-crawl text corpora is only around 5% for many lower-resource languages. |
| Outcome: | The proposed models achieve over 90% average F1 on 1,629 languages . human-judged accuracy for web-crawl text corpora is only around 5% for many lower-resource languages - suggesting a need for more robust evaluation. |
Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution (2023.acl-long)
Copied to clipboard
| Challenge: | Recent results show that annotating mentions is twice as fast as annotation of full coreference chains. |
| Approach: | They propose a method for efficiently adapting coreference models using only mentions in the target domain without increasing annotator time. |
| Outcome: | The proposed method improves average F1 without increasing annotator time. |
ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing guardrails rely on rule-based filtering or single-pass classification, limiting their ability to handle nuanced safety violations. |
| Approach: | They propose a critique-augmented guardrail model that distills knowledge from high-capacity LLMs by generating structured critiques alongside safety labels. |
| Outcome: | The proposed model outperforms existing guardrail models on multiple safety benchmarks and achieves the highest average F1 and AUPRC. |
CancerEmo: A Dataset for Fine-Grained Emotion Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | a lack of large annotated datasets hinders emotion detection in the health domain . a recent study shows that online sharing of emotions is beneficial to a patient's progress . |
| Approach: | They propose an emotion dataset annotated with eight fine-grained emotions from an online health community. |
| Outcome: | The proposed model achieves an average F1 of 71% on the cancerEmo dataset . the best model achieve a higher F1 than the previous model, which was improved using domain-specific pre-training. |
TAMA: Target-Aware Multilingual Abuse Detection by Cascaded Conditional Multi-Task Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for protecting public figures from online abuse ignore who is targeted and how. |
| Approach: | They propose a target-aware multi-task framework that conditions downstream predictions on upstream beliefs via three lightweight modules: Cross-Task Feature Fusion (CTF), Task-Adaptive Gating (TAG), and Label-Guided Span Detection (LGSD). |
| Outcome: | The proposed framework yields higher average F1 than single-task training and standard multi-task learning. |
DiCoRe: Enhancing Zero-shot Event Detection via Divergent-Convergent LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Understanding the complex event ontology, extracting domain-specific triggers from the passage, and structuring them appropriately overloads and limits the utility of Large Language Models (LLMs). |
| Approach: | They propose a divergent-convergent reasoning framework that decouples the task of ED using Dreamer and Grounder. |
| Outcome: | The proposed framework outperforms baselines on six datasets across five domains and nine LLMs, achieving 4–7% average gains over the best baseline. |
SNaRe: Domain-aware Data Generation for Low-Resource Event Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ED struggle with label noise and domain drift when applied to specialized domains. |
| Approach: | They propose a domain-aware synthetic data generation framework composed of three components: Scout, Narrator, and Refiner. |
| Outcome: | The proposed framework outperforms baseline approaches on three diverse domain ED datasets and achieves average F1 gains of 3-7% in the zero-shot/few-shot settings and 4-20% improvement for multilingual generation. |